Precise selection techniques for multi-touch screens

ABSTRACT

A unique system and method is provided that facilitates pixel-accurate targeting with respect to multi-touch sensitive displays when selecting or viewing content with a cursor. In particular, the system and method can track dual inputs from a primary finger and a secondary finger, for example. The primary finger can control movement of the cursor while the secondary finger can adjust a control-display ratio of the screen. As a result, cursor steering and selection of an assistance mode can be performed at about the same time or concurrently. In addition, the system and method can stabilize a cursor position at a top middle point of a user&#39;s finger in order to mitigate clicking errors when making a selection.

RELATED APPLICATION

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 13/295,982, filed on Nov. 14, 2011, titled “PreciseSelection Techniques for Multi-Touch Screens”, which is a continuationof and claims priority to U.S. patent application Ser. No. 11/379,297,filed on Apr. 19, 2006, titled “Precise Selection Techniques forMulti-Touch Screens,” both of which are incorporated by reference hereinin their entirety.

BACKGROUND

In general, touch screen technology has advanced in recent years. Theability to directly touch and manipulate data on the screen withoutusing any intermediary devices has a very strong appeal to users. Inparticular, novices tend to benefit most from the directness of touchscreen displays. A fast learning curve and inherent robustness (nomovable parts) make touch screens an ideal interface for interactingwith public installations, such as information kiosks, automated tellermachines, ticketing machines, retail cashier systems for employee orcustomer use, voting machines, or gambling devices.

While touch screen use has become widespread in such special purposeapplications, its presence in more general computing devices such aspersonal computers and laptops, for example, is far less prevalent. Theslow adoption of touch screens into more general computing devices maybe attributed to known issues of relatively high error rates and armfatigue. In addition, the variable size of human fingers and the lack ofsensing precision can make touch screen interactions difficult at best.

Due to technical restrictions, most commercially available touch screendevices currently in use are only capable of tracking a single point onthe surface of the device. However, multi-touch devices are slowlyemerging into the marketplace. Unfortunately, the multi-touch screensintroduce further challenges in addition to those currently existing insingle-touch screens. For instance, the underlying technology ofmulti-touch sensitive devices often makes their input noisier, thusrequiring further considerations for filtering the undesirable noiseand/or distinguishing the correct input from the noise.

These issues become especially problematic when running programapplications developed for a traditional mouse interface on a multi-usertouch screen. This is primarily because current WIMP (windows, icons,menus, and pointing) user interfaces require frequent selection of verysmall targets (e.g., about 4 pixels or less). For example, window resizehandles are often just 4 pixels wide. Noisy input, lower trackingresolution, and a large potential touch area of a finger tend to createsignificant selection problems.

Furthermore, fingertips can occlude small targets depriving users ofvisual feedback during target acquisition. The user's hands and arms maycontribute to the occlusion problem. Depending on screen orientation,the user may be forced to either look “under hand” (with horizontallypositioned screens) or “over hand” (with angled or vertically positionedscreens). Finally, it is often difficult to decide the optimal point inthe finger's contact area which should anchor the cursor, leaving theusual choice to the center of mass. This can lead to a small butpervasive disconnect between the user's expectations regarding cursorposition and what is actually being sensed and computed. These issueshave been recognized by researchers who have proposed several solutions:adding a fixed cursor offset, enlarging the target area, and providingon-screen widgets to aid in selection. However, these solutions tend tofall short either by introducing new problems or by only improving someproblems and leaving others unresolved. For example, the fixed cursoroffset provides a cursor with a fixed offset above the tip of a fingerwhen the user is touching the screen. Lifting the finger off the screentriggers a selection (“click”). While this method is effective for mosttargets sizes, it has been found ineffective when the target size issmaller than 4 pixels. In addition, the risk or frequency ofunintentional clicks may still be undesirably high.

Others have explored cursor stabilization improvements that effectivelyslow down the cursor movement in various regions around the initialfinger contact point. While this method performed well for the targetacquisition task, precise steering tasks, such as drawing, would be harddue to varying cursor speed. More recently, several on-screen widgetshave been explored for increasing precision while selecting smalltargets on a touch screen. However, their interactions were designed tobe used with touch screens capable of reporting only a single contactpoint and therefore the users were required to execute multiple discretesteps before selecting the target. These steps were delimited by theuser lifting their finger from the screen, thus impeding the overallinteraction performance. Losing overview or context is another maindrawback of this technique which can cause significant problems in manyapplications.

Increasing the relative size of screen targets has also been explored byscaling the display space or scaling the motor space. This workexperimented with hand gestures that activated various levels offish-eye distortion in the interface to facilitate target selection.Techniques that adaptively increase the motor space while leaving thedisplayed image unchanged show promising results without introducingscreen distortions, but require that the system know all targetlocations. This information might not be available in many of today'sapplications. More importantly, such techniques require the use of arelative pointing device such as a mouse. Without such devices, theyintroduce an unpredictable cursor offset when applied directly to anabsolute pointing device such as a touch screen.

Research in the area or multi-touch screens has identified that mostcurrent user interfaces require an interaction model consisting of atleast 3 different states: out-of-range, tracking, and dragging. However,many touch sensitive devices can only reliably sense location in onestate thus making it hard to disambiguate between dragging and tracking(hover). The use of a stylus (pen) is generally preferred in manyinterfaces that require precise interactions. However, while a stylushas a much smaller tip, the associated issues with hand tremor andresolution make the selection task of small targets more difficult thanwith a mouse.

SUMMARY

The following presents a simplified summary in order to provide a basicunderstanding of some aspects of the systems and/or methods discussedherein. This summary is not an extensive overview of the systems and/ormethods discussed herein. It is not intended to identify key/criticalelements or to delineate the scope of such systems and/or methods. Itssole purpose is to present some concepts in a simplified form as aprelude to the more detailed description that is presented later.

The subject application relates to a system(s) and/or methodology thatfacilitate pixel-accurate targeting with respect to multi-touchsensitive displays when selecting or viewing content with a cursor. Morespecifically, target areas for selection, navigation or othermanipulation can be pinpointed with greater precision and accuracy suchas by magnifying the target area, adjusting the speed of the cursor,and/or using various gestures to adjust the position of the cursor.Thus, both cursor steering and selection of an assistance mode (e.g.,cursor offset, scale, speed, or a combination) can be performed at orabout the same time as desired by the user. This can be accomplished inpart by employing dual member (or finger) selection operations. Inparticular, dual finger operations involve tracking multi-touch input,or rather input received from a primary finger and a secondary finger.For example, the primary finger can be used to control the movement ofthe cursor while a secondary finger can adjust the control-displayratio. Thus, the secondary (e.g., non-pointing) finger can quicklymodify or switch cursor manipulation modes without disrupting theprimary (e.g., pointing) finger. Once the precise target selection areais in clear view as desired by the user, the primary finger can make theselection.

In practice, for instance, placement of the secondary finger on thesurface can trigger a number of different events that affect the cursor.The events can be distinguished by various user or screen settings or bydetecting certain movements made by the secondary finger. For example,placing the secondary finger anywhere on the surface of the screen cancause an offset of the cursor from its original position. In thisscenario, either the primary or secondary fingers or both can be used toguide the cursor to its intended target on the screen. Sliding movementsmade by the secondary finger relative to the primary finger can reduceor increase the speed of the cursor depending on the direction of theslide. The amount of the reduction can depend on the distance of theslide. Furthermore, the secondary finger can trigger a menu to appearon-screen, whereby the user can choose an option therefrom (using thesecondary finger) to alter the cursor's speed, position, or offset.

Moreover, the secondary finger can adjust the control-display ratio ofthe screen while the primary finger controls the steering to improvetarget selection and cursor navigation. That is, the primary andsecondary fingers can operate concurrently to provide a fluid andnatural interaction between the user and the content displayed on themulti-touch screen.

Once the cursor is positioned over the desired target, a “click” orselection of the target can be accomplished by using the primary finger.More specifically, the system and method provide for cursorstabilization in order to mitigate unintentional clicks and selectionerrors. Unlike conventional clicking techniques, the cursor position canbe mapped to correspond to a top middle point of the primary finger'scontact area rather than to the center of the contact area or fingermass. Using a rocking motion from fingertip to wrist, the primary fingercan move in this downward manner to signal a click of the underlyingcontent. As a result, more accurate clicks can be performed in arelatively efficient manner.

To the accomplishment of the foregoing and related ends, certainillustrative aspects of the invention are described herein in connectionwith the following description and the annexed drawings. These aspectsare indicative, however, of but a few of the various ways in which theprinciples of the invention may be employed and the subject invention isintended to include all such aspects and their equivalents. Otheradvantages and novel features of the invention may become apparent fromthe following detailed description of the invention when considered inconjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system that facilitates pixel-accuratetargeting with respect to multi-touch sensitive displays when selectingor viewing content with a cursor.

FIG. 2 is a block diagram of a system that facilitates pixel-accuratetargeting with respect to multi-touch sensitive displays that involvesreceiving and processing dual member touch input in order to make cursorselection more precise.

FIG. 3 is a diagram that demonstrates one example of dual member touchinput whereby the secondary finger triggers a change or amount of offsetof the cursor from its original position.

FIG. 4 is a diagram that demonstrates another example of dual membertouch input whereby the secondary finger can stretch and enlarge an areasurrounding the current cursor position (first finger) to facilitateselecting smaller targets with greater precision and accuracy.

FIG. 5 is a diagram that demonstrates another example of dual membertouch input whereby a sliding movement by the secondary finger can alterthe cursor speed upward or downward depending on the direction of theslide.

FIG. 6 is a diagram that demonstrates cursor notification graphics thatcan be employed to indicate the current speed of the cursor (e.g., 4×slower speed (left) to a 10× slower speed (middle) and then to a frozenstate (right)).

FIG. 7 is a diagram that demonstrates another example of dual membertouch input whereby a secondary finger triggers the appearance of a menufrom which an option can be selected in order to adjust the position orspeed of the cursor.

FIG. 8 is a diagram that demonstrates an exemplary user interface of acursor modification menu that has been triggered by a secondary fingerin order to alter the current cursor state.

FIG. 9 is a diagram of an exemplary cursor modification menu including 6selection areas.

FIG. 10 is a block diagram of a system that facilitates selecting orclicking on objects while mitigating unintentional clicks and reducingcursor noise during selection.

FIG. 11 is a diagram that schematically illustrates the system of FIG.10 as it may be employed.

FIG. 12 is a flow diagram illustrating an exemplary methodology thatfacilitates pixel-accurate targeting with respect to multi-touchsensitive displays when selecting or viewing content with a cursor.

FIG. 13 is a flow diagram illustrating an exemplary methodology thatfacilitates pixel-accurate targeting with respect to multi-touchsensitive displays based in part on dual member touch input whenselecting or viewing content with a cursor.

FIG. 14 is a flow diagram illustrating an exemplary methodology thatfacilitates enhancing cursor selection of smaller targets whereby thesecondary finger (member) can stretch and enlarge an area surroundingthe current cursor position (held by the primary finger).

FIG. 15 is a flow diagram illustrating an exemplary methodology thatfacilitates enhancing cursor selection of smaller targets whereby asliding movement by the secondary finger can alter the cursor speedupward or downward depending on the direction of the slide.

FIG. 16 illustrates an exemplary environment for implementing variousaspects of the invention.

DETAILED DESCRIPTION

The subject systems and/or methods are now described with reference tothe drawings, wherein like reference numerals are used to refer to likeelements throughout. In the following description, for purposes ofexplanation, numerous specific details are set forth in order to providea thorough understanding of the systems and/or methods. It may beevident, however, that the subject systems and/or methods may bepracticed without these specific details. In other instances, well-knownstructures and devices are shown in block diagram form in order tofacilitate describing them.

As used herein, the terms “component” and “system” are intended to referto a computer-related entity, either hardware, a combination of hardwareand software, software, or software in execution. For example, acomponent may be, but is not limited to being, a process running on aprocessor, a processor, an object, an executable, a thread of execution,a program, and a computer. By way of illustration, both an applicationrunning on a server and the server can be a component. One or morecomponents may reside within a process and/or thread of execution and acomponent may be localized on one computer and/or distributed betweentwo or more computers.

Current operating systems and program applications such as for wordprocessing, spreadsheets, slide show presentations, drawings, messaging,and calendaring require a high amount of precision for navigation andselection. This is largely due to the fact that when using a touchscreen, operating systems and many of these general computingapplications have a lot of noise in tracking which can affect selectionprecision. As screen resolution increases, selection and navigationprecision naturally increases as well. Although screen resolution formulti-touch screens has improved somewhat in recent years, selection andnavigation precision has not followed to the degree demanded by many ofthese WIMP based applications and operating systems. The subjectapplication as described in more detail with respect to FIGS. 1-15,infra, provide various techniques to enhance the user's experience witha multi-touch screen by allowing the user to modify the control-displayratio when desired (e.g., temporarily) to assist in targeting and toreduce tracking noise while still moving and steering the cursor.

It should be appreciated that the term dual finger as used throughoutthe application refers to the use of any two digits or members includingthose on a user's hand, hands, foot, or feet and is not meant to belimited to only the user's fingers. For purposes of brevity throughoutthe discussion below, the term primary can refer to the first touch orfinger on the surface while the term secondary can refer to the secondtouch or finger detected on the surface. This is based on thepresumption that in general use, the user initially points the cursor tothe desired area on the screen and then desires some assistance withmoving or selecting content with the cursor. For example, the primaryfinger can refer to the finger that the user normally points with whichtends to be the index finger on the dominant hand. The secondary fingercan refer to a helper finger which can be any other finger on the sameor opposite hand. Thus, the terms primary and first and secondary andsecond, respectively, may be used interchangeably. With someinteractions, a single-handed operation is desired. In such cases, thethumb of the dominant hand can serve as a secondary finger.

Referring now to FIG. 1, there is a general block diagram of a system100 that facilitates pixel-accurate targeting with respect tomulti-touch sensitive displays when selecting or viewing content with acursor. The system 100 includes a multi-touch tracking component 110that can track bi-manual or dual finger input. Bi-manual input can beprovided by the use of both hands whereas dual finger input can beprovided from the same hand or through the use of both hands. Inparticular, the tracking component 110 can detect and disambiguatebetween a primary and a secondary finger touch, thereby facilitatingcursor steering (by the primary finger) concurrently with the selectionof an assistance mode (by the secondary finger). By tracking such inputfrom each finger, a cursor control component 120 can regulate the speedor position of the cursor as well as the scale of the content underlyingthe cursor based on the respective inputs. For example, particularmovements made by the secondary finger or the presence of the secondaryfinger on the surface can trigger the cursor control component 120 toautomatically adjust the speed or position of the cursor or thecontrol-display ratio of at least a portion of the content on thescreen.

When the secondary finger is removed from the surface, the contact isbroken which can cause the cursor to return to its previous or defaultstate. Hence, the modifications to the cursor and/or to thecontrol-display ratio can be temporarily invoked by the user on anas-needed basis. By controlling the cursor in this manner, the user canreceive assistance in targeting or in reducing tracking noise whenexplicitly requested. Conventional techniques require users tocontinuously compensate their targeting even when the target is largeenough to be easily selected by direct touch, which can hinder overallmaneuverability and unduly complicate otherwise effortless direct touchinteractions.

Turning now to FIG. 2, there is a block diagram of a system 200 thatfacilitates pixel-accurate targeting with respect to multi-touchsensitive displays that involves receiving and processing dual fingertouch input in order to make cursor selection more precise. The system200 includes an input tracking component 210 that can track anddisambiguate the touch input of at least two fingers. In particular, theinput tracking component 210 includes a primary touch detectioncomponent 220 and a secondary touch detection component 230. The primarytouch detection component 220 can sense when contact is made on thesurface of the screen by a first finger and track its movement or actionuntil such contact is broken (e.g., finger is lifted off or removed fromthe surface). Similarly, the secondary touch detection component 230 cansense a second touch made by another finger, for example.

The input from each detection component can be communicated to an inputanalysis component 240 for processing. The input analysis component 240can examine the touch input as it is sensed by the respective detectioncomponent. The cursor control component 120 can take such analysis andmodify at least one of speed, position, or scale of the cursor accordingto the input, thereby facilitating the selection of desired content. Aselection component 250 can then “click” the desired content.

The detection of the second touch or second finger on the surface cantrigger a variety of events to occur which can temporarily affect thecursor's position, speed, and/or scale. In practice, for example,imagine the user is using a multi-touch screen to view a message. Eachmessage (e.g., in a list of messages) may be a large enough target toaccurately hit without assistance; thus, the user can readily put hispointer finger down onto the screen and select the desired message toopen. Once opened, the user may want to minimize the window for a fewmoments while he views some other content. The “minimize” button can becharacterized as a smaller target (e.g., about 4 pixels or less) thatrequires greater precision during selection. In some operating systems,it can be located next to other window operations such as resize orclose. Hence, less than accurate targeting of the intended button canlead to an undesirable result and can be costly to the user.

To select the “minimize” button, the user can place a first or primaryfinger over or on the minimize button. The first touch detectioncomponent 220 senses this touch and the cursor moves or jumps to thisposition. To obtain a better or larger view of the target area, the usercan place a secondary finger on the screen to the left or right andabove or below the primary finger. The actual placement of the secondaryfinger can depend on the relative location of the primary finger.Following, the secondary finger can be dragged away from the primaryfinger at an angle. The second touch detection component 230 can trackthis movement and automatically invoke a scaled magnification of asquared area surrounding (e.g., centered around) the location of theprimary finger. The initial placement of the secondary finger canidentify the squared area which is to be scaled. Such movements by thesecondary finger can be processed by the input analysis component 240and communicated to the cursor control component 120 in order tovisualize the change in the control-display ratio in this part of thescreen. Hence, the user can adaptively scale a portion of the screenwith the secondary finger to assist in selection precision and accuracywhile still employing the primary finger to perform the actualselection. FIG. 4, infra, further illustrates this selection operation.

Several of the following figures demonstrate alternative approaches toenhancing pixel-accurate targeting. Referring now to FIG. 3, there is adiagram 300 that demonstrates one example of dual finger touch inputwhereby the secondary finger triggers a change in cursor position fromits original position. To provide both variable offset and enable finercontrol of the cursor speed, this technique can be triggered by placingthe secondary finger on the surface 310. The cursor 320 is then offsetto the midpoint between the primary and the secondary fingers. Whileboth fingers are in contact with the screen, moving either or bothfingers controls the movement of the cursor. However, clicking can stillperformed only by the primary finger.

This technique allows for variable reductions in cursor speed: when bothfingers are moving in the same direction and the same speed, the cursorfollows with the same speed; while when only one finger is moving, thecursor moves with half the speed of that finger. This method caneffectively reduce the finger speed by a factor of 2 which yields goodresults for most targets; but it may not provide enough control for thesmallest targets (e.g., 2 pixels or less). An additional shortcoming ofthis technique is that not all locations on the screen are equallyaccessible. For example, screen corners are not accessible usingmidpoint selection.

Generally speaking, the cursor offset is not enabled by default.However, by placing a secondary finger anywhere on the surface 310, thecursor is subsequently offset with respect to the primary finger by apredefined fixed amount. This offset can be programmed to always placethe cursor above the primary finger. Other positions can be chosen aswell. To accommodate both left- and right-handed users, the cursor canbe placed to the left or to the right of the primary finger based on therelative position of the secondary finger. For example, by placing thesecondary finger to the left of the secondary finger to the left of theprimary, the cursor appears to the left of and above the primary finger.

Turning now to FIG. 4, there is a diagram 400 that demonstrates anotherexample of dual member touch input whereby the secondary finger canstretch and enlarge an area surrounding the current cursor position(primary finger location) to facilitate selecting smaller targets withgreater precision and accuracy. As discussed earlier, this stretchingtechnique allows the user to adaptively scale a portion of the screenwith the secondary finger while the primary finger performs theselection. To allow for simultaneous or concurrent “stretching” andselection, the primary finger 410 provides the initial anchor locationaround which the user interface is scaled, while the secondary finger420 identifies the corner of the square area which will be scaled. Inparticular, the secondary finger specifies the square zooming areacentered at the primary finger's location. By moving the secondaryfinger 420 closer or further away from the primary 410 finger, thesquare stretching area is reduced or expanded as illustrated in FIG. 4.Lifting the secondary finger 420 from the table resets the interface toits default un-stretched state. Upon this reset, the cursor is offsetwith respect to the primary finger 410 and is placed where it waslocated in the stretched state. The cursor offset can be reset when allfingers are removed from the table.

The extent of control-display ratio manipulation depends on two physicallimits: the closest perceptible distance between the user's fingers andthe largest diagonal of the screen. For most common mid-screenmanipulations, this stretch technique can enable control-display ratiosroughly up to 10. By allowing clutching and repeated zooming, it can bepossible to further increase this ratio.

Furthermore, this technique has several advantages over conventionaltechniques primarily due to the dual finger design. In such conventionalstrategies, zooming and selection are decoupled into two separateactions, whereas here, they can happen concurrently which results in afluid interaction. In addition, the subject interface scales in alldirections from the primary finger's original location. This provides animportant advantage over traditional rectangle selection where the twopoints specify the diagonal corners of the zooming rectangle (also knownas bounding box zoom).

With traditional rectangle selection, the user tends to place theprimary finger off target in order to capture the target in the zoomedarea; while with the subject technique, the user can place the primaryfinger 410 directly on the target and the interface scales underneath inall directions (as shown in FIG. 4). Placing the finger off-target as isdone conventionally requires the user's primary finger to traverse anincreased distance to perform final selection because the target willappear to move away from the finger as the zoom level increases. Byencouraging placement of the primary finger as close to the target aspossible as is done herein, the eventual distance that this finger willneed to traverse to acquire the target is minimized.

FIG. 5 is a diagram that demonstrates another example of dual membertouch input whereby a sliding movement by a secondary finger can alterthe cursor speed upward or downward depending on the direction of theslide. Given that two-finger interactions are a very natural way ofspecifying distance, this interaction uses the distance between fingersto switch between cursor speed reduction modes. This technique does notpresent an on-screen graphic to the user. Instead, it relies on theuser's ability to gauge the spatial relationship between their fingers.

As illustrated in this exemplary scenario, the right finger (primary)510 controls the cursor and the left finger (secondary) 520 is invokingthis invisible slider control. A cursor notification graphic 530 can beused to signal the cursor speed to the user (see FIG. 6, infra). Movingthe secondary finger 520 towards the primary finger 510 reduces thecursor speed in 3 discrete steps. This allows for the same reductions incursor speed: normal, slow 4×, slow 10×, and freeze (frozen cursormode). Moving the secondary finger 520 away from the primary 510increases the speed up to the normal speed. The distance that thesecondary finger traverses in switching speed reduction modes can bepredefined and is not necessarily dependent on the distance between thefingers.

Continuing to move the fingers apart triggers a “snap” which brings thecursor back to the primary finger's location. Snapping can be signaledby a distinct sound effect to assist the user in recognizing what hasoccurred. Furthermore, the modes can be remembered even after the userlifts the secondary finger which allows for clutching in theinteraction.

In FIG. 6, there is a diagram that demonstrates cursor notificationgraphics that can be employed to indicate the current speed reduction ofthe cursor. For example, the speed of the cursor can be reduced by afactor of J (610), where J is an integer greater than 0, by a factorgreater than J (620), or can be frozen in place (630), thereby disablingany cursor movement. The ability to completely stop the cursor frommoving has at least two benefits. First, by freezing the cursor, theuser can quickly and easily establish a desired cursor offset. This canbe accomplished by freezing the cursor temporarily, moving the finger toachieve the desired offset, and then unfreezing the cursor again.Second, when selecting very small targets, even small amounts of noisecan cause an error. Such noise can be due to device tracking errors,tremor in the user's hand, or noise due to the clicking motion. Byfreezing the cursor in place, the user can ensure that the desiredselection is successful even in very noisy conditions. Moreover, itshould be appreciated that other speed reductions can by employed aswell in addition to or as alternatives to these.

Turning now to FIG. 7, there is a diagram 700 that demonstrates anotherexample of dual member touch input that allows users to adaptivelyadjust the control-display ratio as well as obtain cursor offset whilelooking at an un-zoomed user interface. More specifically, a menu 710(e.g., circular menu) can be invoked whenever the secondary finger 720establishes contact with the surface 730. The menu 710 can be positionedso that the finger 720 is located at its center. The user can select aparticular assistance mode by moving the secondary finger 720 to any ofthe desired regions of the menu. FIG. 8 provides an illustration of themenu as it can be employed in practice. As shown in this figure, theuser desires to select a “close window” button. To increase the accuracyof the button selection, the user can trigger the menu to appearon-screen and then can select a speed reduction level from the menu toslow down the speed of the cursor. As a result, the user can move thecursor over the intended target with substantially more precision andclick on the button with far greater accuracy.

The menu can have six selection areas as shown in FIGS. 7, 8, and 9.Four areas can control the relative speed of the cursor: normal, slow4×, slow 10×, and freeze. Normal mode moves the cursor with the samespeed as the primary finger; the two slow modes reduce the speed of thecursor by a factor of 4 and 10 respectively, while freeze mode “freezes”the cursor in place. The left two areas on the menu can invoke twohelper modes such as “snap” and “magnify”. When snap mode is triggered,the cursor offset (if any) is removed, and the cursor snaps back to thecurrent location of the primary finger. This mode is useful inrepositioning the cursor in the slow movement modes because it is easyto run out of tracked screen space when using the slow cursor modes.

Magnify mode presents a small magnification area in the middle of themenu that shows the enlarged area under the cursor (see FIG. 9, 910).The magnification factor can be fixed at 2×, for instance. This mode isparticularly useful when the primary finger overlaps the cursor. In thiscase, the magnified image acts as a lens showing the portion of theinterface obstructed by the primary finger. As noted earlier, a cursornotification graphic can signal which cursor speed reduction level iscurrently selected, without requiring the user to refer back to themenu. It should be appreciated that the menu can include any number ofselection areas as desired by the system programmer or the user.

Unlike conventional menus, this particular menu does not operate byclicking to make a selection but rather by crossing the finger into aparticular selection area, which enables more experienced users toactivate modes by performing quick strokes in a particular direction.With practice, this selection can be made without looking, and couldtherefore allow for an expert mode in which the menu could be completelyhidden from or made more transparent to the user. Removing the secondaryfinger from the surface can cause the menu to disappear.

Once the cursor is placed over or on the intended and desired target,the user can select the target using the primary finger. Because the actof selecting a target can be noisy due to such factors as trackingerrors, hand tremors, or unintentional movements or flinches,conventional clicking operations tend to be ineffective. For example,such conventional clicking operations typically perform a click when thecontact between the user's finger and the surface is either establishedor broken. These techniques provide a mechanism for clicking but do notaccommodate the needs of current user interfaces that require at least 3different interaction states: out of range, tracking or hovering, anddragging. Both tracking and dragging states require the contact positionto be continuously reported; however, most current touch-sensitivedevices only sense location when the contact is actually touching thesurface, making it difficult to approximate those two states. A possiblesolution is to use pressure-sensing technology and map the increasedpressure to a dragging state, and light pressure to a tracking state.

FIG. 10 illustrates a block diagram of a cursor control component 120that facilitates selecting or clicking on objects while mitigatingunintentional clicks and reducing cursor noise during selection. Thecursor control component 120 does not report pressure directly; howevera pressure-sensitive device can be simulated by employing a mappingcomponent 1010 to map the changes in a finger's contact area to thechanges in pressure. Though applying different finger areas to differentpressure states has been performed in conventional devices, the cursorcontrol component 120 can reduce cursor noise while the user is changingor transitioning between pressure states (clicking). This can beaccomplished in part by stabilizing the cursor movement during clickingvia a cursor stabilization component 1020. In order to effectivelystabilize the cursor during clicking, the user applies a small rockingmotion with their finger in order to perform a “click”, as demonstratedin FIG. 11. Since the user starts pointing with their finger tip (FIG.11, 1110) and then rocks the finger to click (1120), the increase inarea happens predominately in one direction: from the tip point towardsthe user's wrist. As a result, the cursor position (1130) can bestabilized by fixing the cursor location (1130) to the top middle pointof the contact area, rather than to the center of mass as has been donein traditional clicking techniques.

Preliminary experiments have indicated that this point naturally movesmuch less than the center point and therefore reduces the cursor noiseduring clicking. By fixing the cursor to the top-middle point, the useris also able to make a more drastic change in the contact area withoutsignificantly disturbing the cursor location, which aids in reduction ofthe unintentional clicks. Two thresholds on contact area can beestablished to disable spurious switching between the clicking statesdue to noise or hand tremor. An analysis component 1030 (FIG. 10) can beemployed to determine whether the threshold levels have been satisfiedor crossed.

Crossing the high threshold activates the click-on state, and crossingthe low threshold returns back to a click-off state. Due to finger sizedifferences, these high and low thresholds can be automatically ormanually calibrated for each user. In addition, the orientation of thehand and arm can and should be considered since the top-middle point ofthe user's finger will not appear as such if the user contacts thesurface from more than one direction (e.g., straight ahead, from theside, upside-down, etc).

Various methodologies will now be described via a series of acts. It isto be understood and appreciated that the subject system and/ormethodology is not limited by the order of acts, as some acts may, inaccordance with the subject application, occur in different ordersand/or concurrently with other acts from that shown and describedherein. For example, those skilled in the art will understand andappreciate that a methodology could alternatively be represented as aseries of interrelated states or events, such as in a state diagram.Moreover, not all illustrated acts may be required to implement amethodology in accordance with the subject application.

Referring now to FIG. 12, there is a flow diagram illustrating anexemplary method 1200 that facilitates pixel-accurate targeting withrespect to multi-touch sensitive displays when selecting or viewingcontent with a cursor. The method involves tracking multi-touch input at1210. The multi-touch input can include input from a primary finger (ormember) and a secondary finger (or member). At 1220, the control-displayratio can be adjusted by the secondary finger while the primary fingercontrols the movement of the cursor. Thus, these two inputs can beperformed and/or received concurrently or overlapping with respect toone another. Therefore, the cursor can be moved or steered to a desiredlocation while at the same time modifying the type assistance employedto move or steer the cursor.

FIGS. 13-15 present specific methods that can be employed to facilitatepixel-accurate targeting. For example, the exemplary method 1300 in FIG.13 involves detecting a primary or first touch on the surface at 1310.Thereafter, the cursor can be visualized at the location of the primarytouch at 1320. At 1330, a secondary or second touch can be detected onthe surface which can trigger an offset of the cursor based at least inpart on the original position of the cursor. The offset can be removedand the cursor can return to its original location by removing thesecondary finger from the surface.

In FIG. 14, there is a flow diagram illustrating an exemplary method1400 that facilitates enhancing cursor selection of smaller targetswhereby the second contact can stretch and enlarge an area surroundingthe current cursor position (e.g., established by a primary finger). Themethod 1400 involves setting an anchor location based on the primaryfinger's location or position on the surface at 1410. At 1420, a pointon an edge or a corner of a square zoom area can be identified based ona secondary finger's location on the surface. At 1430, content withinthe squared zoom area can be adaptively scaled as the secondary fingermoves away from or closer to the primary finger. For example, moving thesecondary finger closer to the primary finger zooms in on the contentwithin the zoom area, thus making it appear smaller. Similarly, movingthe secondary finger away from the primary finger effectively magnifiesthe content within the zoom area. The primary finger's location can bethe center of the zoom area. By stretching a particular portion of thesurface in this manner, a selection target can be more readily viewedand hit with much greater success.

Turning now to FIG. 15, an exemplary method 1500 is presented thatfacilitates enhancing cursor selection of smaller targets whereby asliding movement by the second contact can alter the cursor speed upwardor downward depending on the direction of the slide. The method 1500involves visualizing a cursor on the surface of a multi-touch screen at1510 at a position or location established by a primary finger, forexample. At 1520, the presence and location of a secondary finger on thescreen surface can be detected. At 1530, the cursor's speed can becontrolled while the cursor is being moved by performing at least one ofthe following: moving the secondary finger closer to the primary fingerlocation to reduce cursor speed an amount based on distance moved; andmoving the secondary finger away from the primary finger location toincrease cursor speed an amount based on distance moved. A maximum and aminimum speed can be set. For example, the maximum speed can beconsidered the normal speed of the cursor—e.g., the cursor in its normalor default state. The minimum speed can be a frozen state, whereby thecursor is not allowed to move until unfrozen.

Though not specifically indicated hereinabove, the subject systems andmethods can be employed in any computing environment where a multi-touchscreen is available. For example, the multi-touch screen may be found ona desktop, laptop, or tabletop device or even other portable deviceswith touch screens.

In order to provide additional context for various aspects of thesubject invention, FIG. 16 and the following discussion are intended toprovide a brief, general description of a suitable operating environment1610 in which various aspects of the subject invention may beimplemented. While the invention is described in the general context ofcomputer-executable instructions, such as program modules, executed byone or more computers or other devices, those skilled in the art willrecognize that the invention can also be implemented in combination withother program modules and/or as a combination of hardware and software.

Generally, however, program modules include routines, programs, objects,components, data structures, etc. that perform particular tasks orimplement particular data types. The operating environment 1610 is onlyone example of a suitable operating environment and is not intended tosuggest any limitation as to the scope of use or functionality of theinvention. Other well known computer systems, environments, and/orconfigurations that may be suitable for use with the invention includebut are not limited to, personal computers, hand-held or laptop devices,multiprocessor systems, microprocessor-based systems, programmableconsumer electronics, network PCs, minicomputers, mainframe computers,distributed computing environments that include the above systems ordevices, and the like.

With reference to FIG. 16, an exemplary environment 1610 forimplementing various aspects of the invention includes a computer 1612.The computer 1612 includes a processing unit 1614, a system memory 1616,and a system bus 1618. The system bus 1618 couples system componentsincluding, but not limited to, the system memory 1616 to the processingunit 1614. The processing unit 1614 can be any of various availableprocessors. Dual microprocessors and other multiprocessor architecturesalso can be employed as the processing unit 1614.

The system bus 1618 can be any of several types of bus structure(s)including the memory bus or memory controller, a peripheral bus orexternal bus, and/or a local bus using any variety of available busarchitectures including, but not limited to, 11-bit bus, IndustrialStandard Architecture (ISA), Micro-Channel Architecture (MCA), ExtendedISA (EISA), Intelligent Drive Electronics (IDE), VESA Local Bus (VLB),Peripheral Component Interconnect (PCI), Universal Serial Bus (USB),Advanced Graphics Port (AGP), Personal Computer Memory CardInternational Association bus (PCMCIA), and Small Computer SystemsInterface (SCSI).

The system memory 1616 includes volatile memory 1620 and nonvolatilememory 1622. The basic input/output system (BIOS), containing the basicroutines to transfer information between elements within the computer1612, such as during start-up, is stored in nonvolatile memory 1622. Byway of illustration, and not limitation, nonvolatile memory 1622 caninclude read only memory (ROM), programmable ROM (PROM), electricallyprogrammable ROM (EPROM), electrically erasable ROM (EEPROM), or flashmemory. Volatile memory 1620 includes random access memory (RAM), whichacts as external cache memory. By way of illustration and notlimitation, RAM is available in many forms such as synchronous RAM(SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rateSDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), anddirect Rambus RAM (DRRAM).

Computer 1612 also includes removable/nonremovable, volatile/nonvolatilecomputer storage media. FIG. 16 illustrates, for example a disk storage1624. Disk storage 1624 includes, but is not limited to, devices like amagnetic disk drive, floppy disk drive, tape drive, Jaz drive, Zipdrive, LS-100 drive, flash memory card, or memory stick. In addition,disk storage 1624 can include storage media separately or in combinationwith other storage media including, but not limited to, an optical diskdrive such as a compact disk ROM device (CD-ROM), CD recordable drive(CD-R Drive), CD rewritable drive (CD-RW Drive) or a digital versatiledisk ROM drive (DVD-ROM). To facilitate connection of the disk storagedevices 1624 to the system bus 1618, a removable or non-removableinterface is typically used such as interface 1626.

It is to be appreciated that FIG. 16 describes software that acts as anintermediary between users and the basic computer resources described insuitable operating environment 1610. Such software includes an operatingsystem 1628. Operating system 1628, which can be stored on disk storage1624, acts to control and allocate resources of the computer system1612. System applications 1630 take advantage of the management ofresources by operating system 1628 through program modules 1632 andprogram data 1634 stored either in system memory 1616 or on disk storage1624. It is to be appreciated that the subject invention can beimplemented with various operating systems or combinations of operatingsystems.

A user enters commands or information into the computer 1612 throughinput device(s) 1636. Input devices 1636 include, but are not limitedto, a pointing device such as a mouse, trackball, stylus, touch pad,keyboard, microphone, joystick, game pad, satellite dish, scanner, TVtuner card, digital camera, digital video camera, web camera, and thelike. These and other input devices connect to the processing unit 1614through the system bus 1618 via interface port(s) 1638. Interfaceport(s) 1638 include, for example, a serial port, a parallel port, agame port, and a universal serial bus (USB). Output device(s) 1640 usesome of the same type of ports as input device(s) 1636. Thus, forexample, a USB port may be used to provide input to computer 1612 and tooutput information from computer 1612 to an output device 1640. Outputadapter 1642 is provided to illustrate that there are some outputdevices 1640 like monitors, speakers, and printers among other outputdevices 1640 that require special adapters. The output adapters 1642include, by way of illustration and not limitation, video and soundcards that provide a means of connection between the output device 1640and the system bus 1618. It should be noted that other devices and/orsystems of devices provide both input and output capabilities such asremote computer(s) 1644.

Computer 1612 can operate in a networked environment using logicalconnections to one or more remote computers, such as remote computer(s)1644. The remote computer(s) 1644 can be a personal computer, a server,a router, a network PC, a workstation, a microprocessor based appliance,a peer device or other common network node and the like, and typicallyincludes many or all of the elements described relative to computer1612. For purposes of brevity, only a memory storage device 1646 isillustrated with remote computer(s) 1644. Remote computer(s) 1644 islogically connected to computer 1612 through a network interface 1648and then physically connected via communication connection 1650. Networkinterface 1648 encompasses communication networks such as local-areanetworks (LAN) and wide-area networks (WAN). LAN technologies includeFiber Distributed Data Interface (FDDI), Copper Distributed DataInterface (CDDI), Ethernet/IEEE 1102.3, Token Ring/IEEE 1102.5 and thelike. WAN technologies include, but are not limited to, point-to-pointlinks, circuit switching networks like Integrated Services DigitalNetworks (ISDN) and variations thereon, packet switching networks, andDigital Subscriber Lines (DSL).

Communication connection(s) 1650 refers to the hardware/softwareemployed to connect the network interface 1648 to the bus 1618. Whilecommunication connection 1650 is shown for illustrative clarity insidecomputer 1612, it can also be external to computer 1612. Thehardware/software necessary for connection to the network interface 1648includes, for exemplary purposes only, internal and externaltechnologies such as, modems including regular telephone grade modems,cable modems and DSL modems, ISDN adapters, and Ethernet cards.

What has been described above includes examples of the subject systemand/or method. It is, of course, not possible to describe everyconceivable combination of components or methodologies for purposes ofdescribing the subject system and/or method, but one of ordinary skillin the art may recognize that many further combinations and permutationsof the subject system and/or method are possible. Accordingly, thesubject system and/or method are intended to embrace all suchalterations, modifications, and variations that fall within the spiritand scope of the appended claims. Furthermore, to the extent that theterm “includes” is used in either the detailed description or theclaims, such term is intended to be inclusive in a manner similar to theterm “comprising” as “comprising” is interpreted when employed as atransitional word in a claim.

What is claimed is:
 1. A system comprising: one or more processors; andone or more memories, storing processor-executable instructions that,when executed by the one or more processors, cause the one or moreprocessors to perform acts comprising: tracking multi-touch input to amulti-touch screen, the multi-touch input comprising an input receivedfrom a primary member and an input received from a secondary member;disambiguating between the input received from the primary member andthe input received from the secondary member; controlling a movement ofa cursor based on the input received from the primary member whilemodifying an assistance mode for manipulation of the cursor based on theinput received from the secondary member, the assistance mode includingadjusting a control-display ratio of a content selected based on theinput received from the secondary member; and invoking a menu includinga plurality of selection areas in response to detecting that thesecondary member has established contact with the multi-touch screen andat least one selection area to adjust a speed of the cursor, theplurality of selection areas comprising at least a magnify selectionarea to invoke a magnify mode that presents a magnified image of contentunderneath the cursor, wherein the magnified image acts as a lensshowing a portion of an interface obstructed by the primary member. 2.The system as recited in claim 1, wherein the primary member overlapsthe cursor.
 3. The system as recited in claim 1, wherein the magnifymode is invoked in response to detecting movement of the secondarymember into the magnify selection area.
 4. The system as recited inclaim 1, wherein the at least one selection area to adjust the speed ofthe cursor comprises: a normal cursor speed selection area to adjust thespeed of the cursor to a normal mode in which the cursor moves at a samespeed as the primary member; and at least one slow cursor speedselection area to adjust the speed of the cursor to a slow mode in whichthe cursor moves at a slower speed than the primary member.
 5. Thesystem as recited in claim 4, wherein the at least one slow cursor speedselection area comprises: a first slow cursor speed selection area toadjust the speed of the cursor to a first slow mode in which the cursormoves at a first reduced speed relative to the primary member; and asecond slow cursor speed selection area to adjust the speed of thecursor to a second slow mode in which the cursor moves at a secondreduced speed relative to the primary member that is different from thefirst reduced speed.
 6. The system as recited in claim 4, wherein the atleast one slow cursor speed selection area comprises a cursor speedselection area to freeze the cursor in place.
 7. The system as recitedin claim 1, wherein the menu further comprises a selection area toreturn the cursor to a current location of the primary member.
 8. Thesystem as recited in claim 1, wherein: the menu includes a circularmenu; and the circular menu is positioned such that the secondary memberis located at a center of the circular menu when the menu is invoked inresponse to detecting that the secondary member has established contactwith the multi-touch screen.
 9. One or more memories, storingprocessor-executable instructions that, when executed by one or moreprocessors, cause the one or more processors to perform acts comprising:tracking multi-touch input to a multi-touch screen based on an inputreceived from a primary member and an input received from a secondarymember; activating a cursor based on the input from the primary member;enabling cursor movement control based on the input received from theprimary member concurrently with adjusting a control-display ratio of acontent selected based on the input received from the secondary member,in response to detecting that the secondary member has establishedcontact with the multi-touch screen; and invoking a menu including aplurality of selection areas in response to detecting that the secondarymember has established contact with the multi-touch screen and at leastone selection area to adjust a speed of the cursor, the plurality ofselection areas comprising at least a magnify selection area to invoke amagnify mode that presents a magnified image of content underneath thecursor, wherein the magnified image acts as a lens showing a portion ofan interface obstructed by the primary member.
 10. The one or morememories as recited in claim 9, the acts further comprising: detectingmovement of the secondary member into a normal cursor speed selectionarea of a plurality of selection areas; and adjusting the speed of thecursor to a normal mode in which the cursor moves at a same speed as theprimary member.
 11. The one or more memories as recited in claim 9, theacts further comprising: detecting movement of the secondary member intoa first slow cursor speed selection area of the plurality of selectionareas; and adjusting the speed of the cursor to a first slow mode inwhich the cursor moves at a first reduced speed relative to the primarymember.
 12. The one or more memories as recited in claim 11, the actsfurther comprising: detecting movement of the secondary member into asecond slow cursor speed selection area of the plurality of selectionareas; and adjusting the speed of the cursor to a second slow mode inwhich the cursor moves at a second reduced speed relative to the primarymember that is different from the first reduced speed.
 13. The one ormore memories as recited in claim 9, the acts further comprising:detecting movement of the secondary member into a cursor speed selectionarea of the plurality of selection areas; and freezing the cursor inplace.
 14. A method comprising: under control of one or more processors,tracking multi-touch input to a multi-touch screen based on an inputreceived from a primary member and an input received from a secondarymember; activating a cursor based on the input from the primary member;and enabling cursor movement control based on the input received fromthe primary member concurrently with adjusting a control-display ratioof a content selected based on the input received from the secondarymember, in response to detecting that the secondary member hasestablished contact with the multi-touch screen; and invoking a menuincluding a plurality of selection areas in response to detecting thatthe secondary member has established contact with the multi-touch screenand at least one selection area to adjust a speed of the cursor, theplurality of selection areas comprising at least a magnify selectionarea to invoke a magnify mode that presents a magnified image of contentunderneath the cursor, wherein the magnified image acts as a lensshowing a portion of an interface obstructed by the primary member. 15.The method as recited in claim 14, further comprising returning thecursor to a current location of the primary member in response todetecting movement of the secondary member into a second selection area.16. The method as recited in claim 14, further comprising invoking amenu comprising a plurality of selection areas in response to detectingthat the secondary member has established contact with the multi-touchscreen, the invoking including presenting the menu as a circular menuthat is positioned such that the secondary member is located at a centerof the circular menu when the menu is invoked.
 17. The method as recitedin claim 14, further comprising adjusting a speed of the cursor inresponse to detecting movement of the secondary member into a selectionarea.
 18. The method as recited in claim 17, wherein adjusting the speedof the cursor includes at least one of: adjusting the speed of thecursor to a normal mode in which the cursor moves at a same speed as theprimary member; adjusting the speed of the cursor to a first slow modein which the cursor moves at a first reduced speed relative to theprimary member; adjusting the speed of the cursor to a second slow modein which the cursor moves at a second reduced speed relative to theprimary member that is different from the first reduced speed; orfreezing the cursor in place.
 19. The system as recited in claim 1,further comprising returning the cursor to a default state in responseto detecting that the secondary member is removed from the multi-touchscreen.